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SUMMARY 


This  report  is  one  of  the  final  in  a  series  on  forecasting  methods 
and  forecast  performance  for  Army  demands  in  the  wholesale  supply  system. 

An  extended  data  base  of  48  quarters  is  used  and  a  summary  of  results  from 
both  old  and  new  forecast  methods  is  presented.  The  analysis  is  more  intense 
than  in  [8];  forecast  algorithms  are  used  on  various  item  activity  classes 
with  the  intent  of  detecting  patterns  which  could  indicate  where  certain 
algorithms  work  best.  It  was  hoped  that  ultimately  a  synthesis  of  procedures 
for  forecasting  by  item  classes  would  be  developed.  However,  the  evaluation 
results  as  found  in  the  report  present  no  clear  cut  improvement  to  the  current 
method  for  active  items.  In  [*f] ,  a  new  forecast  procedure  is  recommended  for 
inactive  items. 
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CHAPTER  I 


INTRODUCTION 


1.1  Overview 

This  report  is  one  of  the  final  in  a  series  on  forecasting  methods  and 
forecast  performance  for  Army  item  demands  in  the  wholesale  supply  system. 

An  extended  data  base  of  48  quarters  is  used  and  both  old  and  new  algorithms 
are  evaluated.  A  more  intensive  analysis  is  made  herein  than  in  [8); 
forecast  algorithms  are  used  on  various  item  activity  classes  with  the 
intent  of  detecting  patterns  which  could  indicate  where  certain  algorithms 
work  best.  It  was  hoped  that  ultimately  a  synthesis  of  procedures  for  fore¬ 
casting  by  item  classes  and  for  Interfacing  across  classes  could  be  developed. 
However,  the  evaluation  results  as  described  in  this  report  present  no  clear 
cut  improvement  to  the  current  method  for  active  items.  In  (*f  ] ,  a  new  fore¬ 
cast  procedure  is  recommended  for  inactive  items. 

1.2  Scope 

The  specific  reasons  and  scope  for  this  report  are  listed: 

a.  IRO  continues  to  maintain  snd  expand  a  data  base  of  aviation 
parts'  demand.  The  data  base  of  report  [3].  quarterly  data  for  9700  items  from 
1967-1973,  has  been  extended  to  a  file  of  13900  items  for  years  1967-1977. 

The  analysis  herein  concentrated  upon  the  years  1971-1977,  the  Vietnam  war 
having  less  impact  than  in  the  earlier  file. 

b.  Several  of  the  algorithms  in  [8]  utilized  forecast  parameters 
(e.g.  Kalman  k-factors)  which  were  obtained  from  statistical  properties  of 
aggregate  time  series  of  demand  and  of  flying  hours  over  that  earlier  7-year 
time  period.  In  this  report  we  update  these  parameters  (as  well  as  compute 
parameters  for  new  types  of  algorithms)  based  on  a  later  7-year  period  1971-1977. 

c.  Reports ,[ 7]  , of  superior  performance  of  moving  average  forecasts 
incorporating  a  Trigg  tracking  signal  by  the  TARCOM  Systems  Analysis  Group 
led  IRO  to  modify  such  a  Trigg  algorithm  and  to  test  it  with  the  expanded 
data  base. 

d.  Preliminary  success  of  the  Trigg  tracking  signal  led  IRO  to 
develop  other  refined  algorithms  which  utilized  other  switching  signals  for 
fixing  the  length  of  past  history  used  in  a  current  forecast. 
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e.  Changes  have  been  made  in  the  supply  simulator  since  it  was  used 
as  one  of  the  performance  measuring  tools  in  [s ] •  Additional  improvements 
developed  during  the  course  of  the  work  have  been  included  in  the  final 
simulation  analyses. 

1.3  Findings 

For  the  cursory  reader,  the  table  on  page  6  is  probably  the  best  culmina¬ 
tion  of  the  many  results  and  tables  in  the  body  of  the  report.  At  a  glance 
one  can  see  the  basic  breakout  of  the  analysis  per  stratification  class.  (See 
bottom  of  page).  A  95%  confidence  interval  was  plotted  for  the  mean  difference 
in  performance  (as  measured  by  the  simulator)  per  item  between  the  alternative 
algorithms  and  standard  (1794)  for  the  indicated  stratification  classes.  In 
each  case,  none  of  the  alternative  algorithms  performed  significantly  better 
than  the  standard  (each  confidence  interval  contains  zero). 

The  final  candidate  forecast  algorithms  considered  for  this  analysis 

were: 

1794:  the  current  8  quarter  moving  average 

KAL:  a  modified  Kalman  filter  (see  [9])  which  is  similar  to 
exponential  smoothing. 

XROTRIGG:  a  switching  scheme  between  an  8  quarter  and  4  quarter  moving 
average,  with  some  extra  weight  given  to  the  current 
quarter's  demand. 

MED4:  a  4  quarter  moving  median. 

MDVD:  a  8  quarter  moving  average  on  demands  only. 

It  should  be  noted  that  these  findings  are  limited  to  the  more  active  classes 
of  items.  Report  [H]  has  been  written  which  deals  specifically  with  items 
exhibiting  low  demands. 


YEARLY  DOLLAR  DEMAND 
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CHAPTER  II 


DATA 

2.1  Description 

The  IRO  demand  history  file  includes  11  years  of  requisitions  and  demand 
by  quarter  accumulated  from  the  AVSCOM  Demand  Return  &  Disposal  (DRD)  files 
from  1967  thru  1977.  Flying  hours  covering  the  same  period  as  the  demand 
data  were  obtained  from  DCSLOG.  The  file  contains  a  sample  of  20,865  items 
from  all  those  in  the  system  in  1966  and  subsequently  entered. 

The  data  base  is  limited  to  recurring  demands  for  which  program  data 
was  available.  SSA  and  Grant  Aid  demands  were  eliminated  as  were  items  not 
purchased  thru  central  procurement,  based  on  the  last  recorded  IMPC  code. 

Every  attempt  was  made  to  drop  items  subject  to  logistical  transfer. 

Previous  IRO  forecasting  projects  used  an  older  7-year  data  base  [3], 
compiled  in  much  the  same  way  from  the  DRD  files.  The  only  significant 
difference  in  the  new  data  base  is  the  inclusion  of  items  with  trivial  demand, 
essential  for  forecasting  demand  for  inactive  items. 

2.2  Classification 

Each  item  was  classified  as  low  dollar  value  (LDV)  or  high  dollar  value 
(HDV)  according  to  whether  the  demand  rate  averaged  over  the  11  years  was 
less  than  or  greater  than  or  equal  to  $50,000;  and  the  requisition  rate  was 
less  than  or  greater  than  or  equal  to  100  per  year.  Items  with  over  a  million 
dollars  of  demand  per  year  were  dropped. 

The  items  were  further  divided  into  dynamic  (DYN)  and  non-dynamic  (NON) 
based  on  the  Federal  Stock  Class  (FSC) .  The  dynamic  components  were  considered 
to  be  those  that  experience  high  rotation  rates;  i.e.  rotor  blades,  trans¬ 
missions,  and  turbine  engines.  For  more  detail  see  Cohen  [3]. 

The  data  breaks  out  into  the  following  four  groups: 


HDVDYN 

86 

HDVNON 

262 

LDVDYN 

1169 

LDVNON 

19348 

20865 

Some  of  the  simulation  and  other  forecasting  work  was  done  using  the 
last  7  years  of  the  11  year  data  base,  thus  eliminating  the  1967-1970  period 


subject  to  the  Vietnam  war.  LDV  and  HDV  divisions  were  recreated  based  on 
the  demand  in  the  last  7  years.  The  new  breakdown  of  items  for  this  period 
1971-1977  is: 


HDVDYN 

54 

HDVNON 

224 

LDVDYN 

1199 

LDVNON 

19384 

20861 

Four  items  were  eliminated  which  had  over  one  million  dollar  demand,  based  on 
the  last  7  years. 


Jreeate  Series 


The  graph  on  page  9  illustrates  the  aggregate  series  of  all  the  items 
for  demand  (D) ,  flying  hours  (H) ,  and  demand  divided  by  flying  hours  (D/H) . 


CHAPTER  III 


KALMAN  FILTERS 

3.1  Basic  Forecast  Methods 

As  noted  in  references  [8],  [9],  "Kalman  Filters"  are,  for  a  general  class 
of  statistical  processes,  optimal  forecast  procedures,  in  the  sense  of  mini¬ 
mizing  mean  square  forecast  error.  These  algorithm.''  have  a  general  exponential 
smoothing  structure  where  the  smoothing  weights  themselves  are  variable  and 
updated . 

In  this  section  the  most  important  Kalman  algorithms  that  were  tested 
are  briefly  summarized  and  designated  with  code  names.  Complete  mathematical 
descriptions  are  found  in  the  Appendix  to  this  chapter.  All  algorithms  operate 
on  the  demand  per  flying  hour  time  series  D/H  in  order  to  predict  a  rate  value. 
Forecasts  for  demand  in  a  future  period  are  then  made  by  multiplying  the  rate 
estimate  D/H  by  the  program  (flying  hours)  for  that  future  period. 

KALMAN  -  Original  Kalman  filter  algorithm  investigated  in  reference  [8]. 

I’pdated  k  factors  for  D/H  for  the  period  1971-1977  are 
used . 

KALNEW1  -  Kalman  filter  with  a  mod  if  ■*  cation  to  the  formula  for  updating 
the  weights.  New  factors  are  used.  (See  Section  3.2) 

KALMANS  -  A  switching  signal  is  used  to  choose  either  the  current 
observation  of  D/H  or  the  KALMAN  estimate. 

KALMANSMA2  -  As  in  KALMANS  but  replace  current  observation  by  current 
2  quarter  average. 

KALNEW1S  -  As  in  KALMANS  but  uses  the  KAuNFWl  estimate. 

KALNEW1SMA2  -  As  in  KALNEW1S  but  replace  current  observation  by  current 

2  quarter  average. 

KALNEW1SMA3  -  As  in  KALNEW1S  but  replace  current  observation  by  current 

3  quarter  average . 

KALREL  -  Current  weighting  value  in  original  KALMAN  procedure  is 

adjusted  by  a  "relevance"  function  which  tends  to  put 
more  weight  on  current  observations. 

KAL2SPK  -  An  a  priori  two  spike  distribution  on  zero  value  and  a  non¬ 
zero  value  for  k^^  is  assumed.  Bayesian  updating  is 
applied  to  the  spike  probabilities  and  the  estimates  from 
2  Kalman  filters  (for  k  ■  0  and  k  f  0)  are  appropriately 
weighted.  10 


3.2  K-factors  for  Kalman  Algorithms 

A  parameter  is  needed  to  obtain  the  changing  values  of  weights  applied 

to  past  and  current  observations  of  a  time  series  variable  y  in  a  Kalman 

forecasting  procedure.  The  details  on  the  mathematical  theory  of  Kalman  k 

factors  and  on  the  statistic  formed  from  empirical  forecast  errors  can  be 

found  in  [9].  Suffice  it  here  to  say  that  the  mean  square  error  on  y  over  L 

periods  when  using  a  moving  average  of  B  periods  is  a  function  of  k  and  of  a 
2 

variance  q  of  the  process  mean.  The  functional  relations  can  be  solved  to 
find  estimates  of  k. 

The  tables  present  the  average  k  values  for  D  and  D/H  processes  for 
items  falling  in  various  requisition  classes.  The  1967-1973  table  is  a  re¬ 
finement  of  the  table  for  that  same  period  in  [8],  a  result  of  correcting 
a  not  completely  innocuous  bias  in  data  processing;  k^  did  not  change  much, 
but  kp^  now  does  not  increase  continuously  as  requisition  activity  goes  up. 
The  period  1971-1977  indicates  generally  higher  values  of  k^,  k^^;  hence 
relatively  more  stability  in  the  processes  D,  D/H  than  in  the  Vietnam  era. 
There  has  also  been  a  shift  in  the  column  trends  in  the  later  era,  an  indica¬ 
tion  of  a  change  in  demand  patterns  by  requisition  class.  The  1971-1977  k 
values  are  probably  more  representative  of  ''normality"  but  should  be  updated 
about  every  three  years  or  with  changes  in  war  or  economic  environment. 


3 . 3  Appendix  A  -  Mathematical  Addendum 


Basic  Notation 


y^  *  observed  value  of  process  in  period  n. 

For  all  algorithms  listed,  Demand/Flying  Hours,  D/H 

was  the  utilized  observation  variable  y  in  the  empirical 
analysis. 

■  mean  of  process  in  period  n. 
x^  ■  estimate  of  mean  of  process  at  end  of  period  n. 

y^Gl)  *  forecast  at  end  of  period  n  of  the  process  value  !  periods 
later. 

Except  for  a  few  cases  with  assumed  deterministic  trend 
components  in  the  process  model  y  (which  did  not  perform 
well)  ,  our  processes  assume  stochastic  fluctuations  in 
the  future  around  the  current  mean;  therefore  a  best  MSE 
forecast  “y  (1)  ■  ^  is  used  in  the  cases  below. 
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flying  hours  in  future  period  n+<. 


H(n,L)  =  total  flying  hours,  period  n+1  to  n+L . 

A 

D(n,L)  *=  forecast  of  demand  over  L.  periods  based  on  information  thru 
end  of  period  n 
n+L 


H(n,L) 

n 


KALMAN 


x  »  "x  .  +  G  (y  -  x  ,) 
n  n-1  n  n  n-1 


i  +  k  G 


9  9  V"*-/ 

1  +  k  G  +  k  (H  /H~  ) 

n  n  n+1 

y  (O  =  x 

n  n 

k  is  updated  every  4  quarters  based  on  the  current  8  quarter  moving 
average  estimate  of  yearly  requisitions.  (Table  lookup) 


KALNI'.Wl 


As  above  with 

i  +  k  c  orL/m 

„  n  n+1  n  , 

G  «  - , - = -  (A3) 

1  +  k  G  (H  ,/H  )  +  k 
n  n+1  n' 

KALMANS 

In  reference  [  ]  the  formula  for  the  MSL  of  a  one  period  (quarter) 
forecast  for  moving  average  of  M  periods  applied  to  the  Dynamic  Mean  process  is 

MSI  -  q2[k  +  k/M  +  Witt?*!),  j 

■M  q  1  '  6M  J  (A  4) 

So  in  terms  of  two  moving  average  mean  square  errors,  solving  (A4)  for  k, 

„  l"sV  g-fov  s) 

?2±l  -  c  c  ) 

'  N  '‘‘M  '  M  N' 


where  CT  -  1  +  K  / 

I  ol 

For  N  =  4,  M  -  8  and  R  -  MSE/MSE^ 

Then  (AS)  leads  to 

k  i  .  r>  If  and  only  if  R  Z.  1.5  (A6) 

We  approximate  the  effect  of  k  inferred  to  be  quite  snail  by  putting  all 

weight  (G  -1)  in  a  Kalman  algorithm  on  the  current  observation  y  . 

n  14  n 


The  above  relations  lead  to  the  following  heuristic,  hybrid  algorithm 
employing  a  switching  signal. 

Let 

Zn(J)  «  j  quarter  moving  average  after  period  n. 

MSE  (j)  -  (l-t»)  MSE  .(j)  +  a  (Z  ,(j)  -  y)2  (A7) 

n  n~i  n-i  n 


Then 

if  [MSEN(8)/MSEn(4)  >  1.51^00  -  yn 

otherwise  'y  ( l )  «*  'x  «  x  ,  +  G  (y  -  x  ,) 
n  n  n-1  n  Jn  n-1 

where  G  is  given  by  (A2). 
n 

a  was  chosen  to  be  .8  based  on  empirical  testing. 

KALMAN SMA2 

The  reasoning  applied  here  to  obtain  a  heuristlcal  switching  signal 

algorithm  is  similar  to  that  of  KALMANS.  We  wish  to  know  when  it  is  appropriate 

to  use  an  MA2(2  quarter  moving  average)  on  y  (i.e.  D/H)  . 

Note  in  (A2)  ,  that  when  flying  hours  are  stable  (H  ~  H  G 

n  n+i 

approaches  .67  for  k  ■  .7.  An  exponential  smoothing  weight  of  .67  is  equivalent 
(in  the  sense  of  average  weighting  of  all  past  history)  to  a  moving  average 
base  B  *  2. 

For  N  -  2,  M  -  8  and  R  =  MSEg/MSE^ ,  then  (A5)  leads  to: 

k  <_  .7  if  and  only  if  R  >_  1.728  (A8) 

Then  using  the  notation  of  (A7) ,  define  KALMANSMA2  as 

y  +y  , 

If  [MSE  (8)  /MSE  (2)  >  1.728]  3,tt)  “  ?  y”-- 
n  n  n  i 

Otherwise  y  (1)  -  x  ■  x  .  +  G  (y  -  ^  ) 

n  n  n-i  n  n  n-i 

where  Gr  1b  given  by  (A2) 

KALNEW1S 

Use  Gr  computed  via  (A3)  in  KALMANS  procedure. 
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KALNEW1SMA2 


Use  G  computed  via  (A3)  in  KALMANSMA2  procedure, 
n 

KALNEW1SMA3 

Since  k  »  2  implies  steady  state  C  *  .5,  which  in  turn  indicates 
a  moving  average  base  B  *=  3  (see  reasoning  in  KALMANSMA2) ,  equation  (A5)  is 
used  with  N  •  3,  M  «  8  to  find 


k  12  if  and  only  if  MSE./MSE  >  1.288 

a  J 

Hence  the  following  heuristic  for  KALNEW1SMA3 : 


if  [MSE  (8) /MSE  (3)  >  1.288]  y  (l) 
n  n  n 


y„  +  t  +  y„  , 

n  n-i  n-z 


Otherwise  y  (*)  -  'x  =  x  ,  +  G  (y  -  'x  .) 

n  n  n-i  n  n  n-i 


where  G^  is  given  by  (A3) 


KALREL 


1  +  A 


,  r  =  1 


where 


1  +  k  G 
1  +  k 


(A10) 


If  0  <_  r  <  1 ,  then  more  weight  is  applied  to  the  observation 

y  ...  The  factor  r  can  be  defined  as  the  value  of  a  relevance  function, 
n+1 

which  measures  in  some  manner  the  relevance  of  the  current  observation. 
Extending  the  theory  of [3a]  to  our  models,  a  relevant  relevance  function 


may  be  formulated,  viz.. 


2.5  C 


v  (  - _ SHIP  ...  \ 

r  -  min  jl.  ^  k  H2  ) 

/  x  n 


(i  +  k  r.  ♦  k  h2/h2  )  N|  ... ., 

n _ n  n-tj  (All) 


'V  y 

where  q^  *  standard  deviation  of  demand  mean  based  on  items  requisition 
class.  See  Table 

2 .  S  Is  an  adjustable  value  which  here  bounds  the  difference 

A. 

between  r  he  new  -  stlmate  x  and  the  observation  to  hi-  no 

more  than  .  r>  standard  deviations  of  the  process. 
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This  procedure  hedges  towards  y  ,  as  a  new  estimate  if  the  error 

/N 

magnitude  |y  ..  -  x  I  is  large. 

1  n+i  n 1 

KAL2SPK 


A  two-spike  distribution  for  the  item's  k-factor  is  assumed  -  around 
values  k  -  0  and  k  ■  k^^,  where  the  latter  is  tabulated  by  the  item's  requisi¬ 
tion  class.  The  probabilities  zn  of  k  •  k^yH  are  updated  as  below  and  used  in 

combining  the  Kalman  estimate  “x  from  (Al)  for  k  4  0  with  y  ,  the  Kalman  estimate 

n  n 

for  k  «  0,  i.e. 


9na) 


z  x  +  (1-z  )y 
n  n  n  7n 


(A12) 


If  w^,  are  proportional  to  the  probabilities  p  (error  in  period  n/k)  for 
k  -  k^jj  and  k  ■  0  respectively  and  assuming  normality  of  errors,  then  it  can 
be  shown  for  the  Dynamic  Mean  model. 


"  -$r~  exp !  ~  Cvn  ~  Vi)2/2  BV 

'n  *  «*P  V 


(A13) 

(A1A) 


where 


^2  2  ^2 

Vn  -  q_/H  ,  ,  q_  ■  tabulated  variance  of  demand  mean, 
u  D  n-i  1) 


B  -  (1  +  k 


n-1 


+  k  Gn-1> 


BVq  -  theoretical  variance  of  error. 

Straightforward  application  of  Bayesian  updating  yields  new  z 


z  ,w 
n-1  n 


z  ,w  +  (1-z  ,)v 

n-1  n  n-1  n 


(A15) 


REGKBNEVSMA2 


This  algorithm,  see  [9],  is  the  closest  weighted  average  analogue  to 
a  Kalman  filter's  weighting  of  past  history;  in  addition  a  switch  is  Incorporated 
ala'  KALMAN SKA 2 .  Hence  referring  to  the  section  of  KALMAN SMA2 
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if  [MSE  (8) /MSE  (2)  >  1.7281  y  (i) 

n  n  J  n 


y  +  y  . 
Jn  n-1 


otherwise 

yn<*> 


B 

£ 


j=l 


Wn-j+lyn-l+l 


where 


w 


n-j+1 


H 


2 

n-j+1 


H  j  1 1 

n-i+1 


(A16) 


Base  period  B  is  determined  from  the  nearest  integer  [_» 


-  •  — -  \ 

B.' 

\ 

N  2 

L  4 

(A17) 


and  k  is  updated  every  A  quarters  based  on  the  current  8  quarter  moving 
average  of  yearly  requisitions  (Table  lookup  on  k^^)  . 


3 . A  Appendix  B  -  Structural  Changes  in  KALMAN  to  obtain  KALNEW1 

With  a  slight  change  in  the  assumed  process  model,  the  Dynamic  Mean  model 
(see  [9]  Chapter  II),  the  updating  formula  for  the  weight  In  period  n+1  applied 
to  the  current  observed  value  yn+-^  of  the  process  becomes 


2  2 

q  . .  +  r  G 
n+1  n  n 


n+1 


2  2  2 

q  . .  +  r  G  +  r 
n+1  n  n  n+1 


(Bl) 


2 

instead  of  equation  2.5  in  [9].  Tf  as  in  [9]  Chapter  IV,  it  is  assumed  that  r  , 

2  2  2  n 

the  variance  of  y  r  D  /H  ,  varies  with  l/H^,  and  that  r  /q  ■  ,  then 

yn  n  n  n*  n  n  D/H 


1  *  <"ll/Hn>  Vk  G„ 

1  +  Wn  +  S/H 


as  opposed  to 


G 


n+1 


1  +  k  G 

n 


1  +  k 


G 

n 


+  k  H2/H 
n 


2 

n+1 


(B2) 


(B3) 


as  given  in  [  ] . 

Equation  (B2)  is  the  basis  for  the  KALNEW  algorithms.  Note  as  k  becomes 
large  (B2)  and  (B3)  behave  similarly. 
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CHAPTER  IV 


IRQ  TRIGG  TRACKING  SIGNAL 

4.1  Background 

The  IRO-TRIGG  is  a  modified  version  of  the  TACOM  New  Parameter  method 
described  in  [?].  It  utilizes  a  TRIGG  tracking  signal  to  determine  whether 
the  series  is  stationary  (constant  mean)  or  not  and  uses  an  adjusted  eight  or 
four  quarter  moving  average  respectively  as  the  forecast. 

The  tracking  signal  is  the  ratio  between  a  weighted  average  of  the 
algebraic  (signed  forecast)  errors  and  absolute  (non  signed)  error,  while 
using  a  standard  eight  quarter  moving  average  (MA8)  as  the  forecast.  If  the 
series  is  stationary,  the  MA8  will  do  well  in  forecasting  the  constant  mean 
of  the  system,  and  the  over  and  under  forecast  errors  will  be  on  average 
algebraically  sum  to  zero.  On  the  other  hand,  if  the  series  is  following  a 
trend,  then  the  MA8  will  always  underforecast  (trend  positive)  or  over  forecast 
(trend  negative)  and  thus  the  algebraic  sum  of  the  errors  will  differ  by  the 
absolute  sum  only  in  sign. 

Thus  if  the  signal  is  close  to  ±1,  then  the  series  is  following  a  trend 
whereas  if  the  signal  is  close  to  0,  then  the  series  is  stationary  with  a 
constant  mean. 

4.2  Computation 
Let: 

MA8(n)  *  1/8  I  Y . ,  where  Y,  *  Demand  -  j_n  quarter  i 

i  i  Flying  Hours 

n 

MA4(n)  -  1/4  l  Y 
i-n-3 

AMA8(n)  -  a,Y  +  (l-a.)MA8(n)  (0  <  a.  <  1) 

in  i  —  i  — 

AMA4(n)  *  a„Y  +  (l-o_)MA4(n)  (0  <  a.  <  1) 

£  n  £  —  £  — 

ERROR (n)  -  Y  -  MA8(n-l) 
n 

MALE (n)  -  oERROR(n)  +  (l-a)MALE(n-l)  (0  <  a  <  1) 

(Mean  Algebraic  Error) 

MABE(n)  -  a /ERROR (n)/  +  (l-a)MABE(n-l) 

(Mean  Absolute  Error) 
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TS  (n)  =  MALE  (n)  /MABE  (n) 

(Tracking  Signal) 

Forecast 

\+i  =  if  |Ts(n)|  <  e  (o  <  b  <  i) 

\  AMAA (n)  if  | TS(n> )  >  8 
Initialization  for  Tracking  Signal 
Ref  [  ] . 

Let :  ^ 

MA7 (7)  =  1/7  E  Y 
1 

ERROR (8)  =  Y  -  MA7 (7) 

Q 


MALE(7)  =  0 

7 

MABE (7)  =(l.25/7)E/Yi  -  MA7(7)/ 


Then 

TS  (8) 


a  ERROR (8) _ 

a  | ERROR (8) |  +  (l-a)MABE(7) 


4 • 3  IRO-TRIGG  Parameters 

The  IRO-TRIGG  forecast  procedure  depends  on  certain  parameters  which  have 
to  be  determined  prior  to  the  forecast.  These  parameters  may  be  selected 
subjectively  or  empirically  by  experimentation.  Remarks  and  observations 
concerning  these  parameters  are  as  follows: 

Remarks 

a.  The  and  parameters  define  the  weights  given  to  the  past 
observation  when  computing  AMA8(n)  and  AMA4(n),  i.e. 

n-1 

AMA8  (n)  ■=  W.  £  Y  +  W„Y 
X  _  jL  *L  n 
n-7 

where  =  1/8  (1-a^) 

and  W2  =  1/8  (1+7  ®  ) 

Hence  as  0,  AMA8(n) 

1,  AMA8 (n) 
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-*  MA8 
->  Y 

n 


and  as 


“1  + 


* 

n- 

-1 

* 

AMA4(n)  -  W 

I 

Y 

.  +  W,  Y 
i  2  n 

n- 

-3 

where 

* 

W1 

= 

1/4 

(1^2) 

and 

* 

W2 

- 

1/4 

(1-3  a2) 

also  as 

°2 

-*■ 

o, 

AMA4  (n)  -*  MA4 

“2 

-V 

1, 

AMA4  (n)  -»■  Y 

n 

So  if  <*2  is  small  and  is  large 

then  the  AMA8(n)  will  do  better  then  AMA4(n)  in  forecasting  a 
non-stationary  (trend)  series , 

also  AMA4(n)  would  do  better  then  AMA8(n)  in  forecasting  a  stationary 
series. 

Since  this  is  counter  to  the  logic  of  the  switching  process,  careful 
judgement  should  be  made  when  selecting  the  combination  of  values  for  and  ct^. 

b.  The  a  parameter  determines  the  amount  of  weight  given  to  the 
latest  error  when  using  MA8  for  computing  the  tracking  signal. 


TS (n)  -  - 

(Tracking  Signal) 

Now  as  a  -*■  1,  TS(n) 


a  ERROR  +  (1-a)  MALE(n-l) 
a [ERROR I  +  (1-a)  MABE(n-l) 


ERROR 


+  1 


| ERROR | 

independent  to  the  type  of  series  being  forecasted.  Therefore  for  a  large, 

the  TRIGG  signal  may  incorrectly  identify  a  trend  condition. 

c.  The  g  or  threshold  parameter  Identifies  the  region  in  which 
the  tracking  signal  indicates  a  stationary  or  non-stationary  series.  The 
larger  the  g  the  more  confidence  is  given  to  correctly  identifying  a  non- 
stationary  series,  but  less  confidence  is  given  to  correctly  identifying  a 
stationary  series.  From  the  previous  paragraph,  it  is  obvious  that  a  and  g 
are  related  and  that  as  a  gets  large  6  should  likewise  get  large  in  order  to 
maintain  the  same  level  of  confidence. 
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Observations :  (Results  from  experimentation  with  data  base) 

a.  We  found  that  =  -15  worked  as  well  or  better  then  any  other 

values  selected  for  experimentation  and  these  values  were  somewhat  robust. 

b.  The  best  a  level  was  found  to  be  .66,contrary  to  the  small  implied 
in  the  remarks.  The  switching  process  was  very  sensitive  to  changes  in  this 
parameter.  For  a  =  .66,  the  tracking  signal  indicated  a  non-stationary  process 
74%  of  the  time. 

c.  It  was  surprising  that  changes  in  the  8  level  made  little  difference 
in  the  statistical  results.  6  of  .5  was  choser . 

d.  The  IRO-TRIGG  with  parameters^  =  =  .15,  a  =  .66,  and  8  =  .5 

was  the  final  model  chosen  to  represent  the  tracking  signal  technique.  This 
method  performed  better  then  the  other  alternatives  not  listed  in  the  report. 

4 . 4  Structural  Change  tc  the  TACOM  New  Parameter  Algorithm 

As  mentioned  earlier  the  IRO-TRIGG  forecast  algorithm  is  a  modified  version 
of  the  TAG OK  New  Parameter  algorithm  cited  in  [7].  The  changes  as  described 
below  were  made  in  an  effort  to  make  the  forecast  technique  more  consistent  with 
the  underlying  theory  of  the  TRIGG  tracking  signal. 

a.  Computing  the  Tracking  Signal,  instead  of  using  the  errors  from  the 
actual  forecast  as  done  by  TACOM,  the  IRO  version  uses  the  errors  from  an  eight 
quarter  moving  average  forecast  in  computing  the  tracking  signal.  By  so  doing 
the  tracking  signal  becomes  a  monitor  for  trends.  The  eight  quarter  moving 
average  will  lag  behind  any  trend  in  the  data  which  will  result  in  a  tracking 
signal  close  to  +  one.  Since  the  tracking  signal  is  computed  independently 

of  the  actual  forecast,  it  is  not  effected  by  any  switch  in  the  forecast 
technique  and  will  continue  to  indicate  a  trend  as  long  as  there  is  one. 

b.  q^  and  q^  Constraints.  The  TACOM  New  Parameter  algorithm  utilizes 

two  empirically  found  parameters  q^  and  which  adjust  the  amount  of  weignt 

given  to  the  current  observation  in  both  the  eight  and  four  quarter  moving 
average  computations (0  <_  1)  ,  (0  <*2  <_  1)  .  The  IRO-TRIGG  version  also 

uses  these  parameters  but  constrains  their  values  so  that  the  adjusted  four 
quarter  average  still  responds  faster  to  changes  (trends)  in  the  data  than 
the  adjusted  eight  quarter  average.  (This  will  not  be  the  case  if  is 
sufficiently  large  and  q^  is  sufficiently  small  as  noted  under  remarks, 
section  a  . )  . 
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c.  An  Additional  Empirical  Parameter  a.  When  computing  the  TRIGG 
tracking  signal,  an  exponentially  smoothed  algebraic  error  is  compared  with 
an  exponentially  smoothed  absolute  error.  The  TACOM  New  Parameter  algorithm 
uses  a  smoothing  constant  of  .66  for  these  calculations.  The  IRO-TRIGG 


version  considers  this  a  smoothing  constant  as  an  empirical  parameter  which 
needs  to  be  estimated. 


CHAPTER  V 


CURRENT  AND  OTHER  FORECAST  METHODS 


5.1  Current  -  1794 


The  current  Army  method  of  forecasting  estimates  the  demand  per  program 
(flying  hours)  at  end  of  period  n  using 


'x  =  E  D 


8 


/£  H 

n  n-j+l'.  .  n-J+1 

j=l  J  j  =  l  J 


This  may  be  written  in  terms  of  a  weighted  moving  average  on  y  =  D/H: 

8 

x  =  Z  W  Y  ... 
n  j=1  n-j+l  n-j+l 


W 


n-j+l  8 


"n-j+l 


E  H  -ii 

i=l  n_1  1 


5.2  Moving  Median  (MED4) 

In  an  effort  to  eliminate  the  effects  of  spikes  In  the  data,  a  simple 
four  quarter  moving  median  on  D/H  as  described  below  was  used. 

Let  s(n)  =  >y  . ,  y  ,, ,  y  ,,  y  j  be  the  last  four  observations 
L  n-j  n -z  n-l  n  \ 


and  let 


/•y,  »  y„  »  y,  .  y,  \  b0  the  ordered  array  of  this  set 

\  l  ,n  l  ,n  j  ,n  4  ,n; 


where  y  £  y  for  i  <  j 
i»n  j,n 


Then 


and 


y,  „  +  y-,  „ 

MED4  (n)  =  — ~t~— x — ^ 


^+1  =  MED4  (n) 


Several  data  experiments  were  performed  on  the  length  of  the  base  period 
and  the  four  quarter  base  appeared  to  perform  best. 


A  period  ■  one  quarter  of  a  year,  so  8  quarters  or  2  years  of  history  is  used. 
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5.3  Moving  Average  on  Demand 

To  support  Cohen's  [3]  findings  that  the  use  of  program  data  (flying  hours) 
improves  forecasts  a  simple  eight  quarter  moving  average  was  applied  to  the 
demand  series  D. 

Let  \x  —  j  x  \  be  the  latest  eight  observation 

n-/  n— o  n  j 


x  .  /8 
n-x 
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CHAPTER  VI 


EXPERIMENTAL  DESIGN 

6.1  Overview 

From  our  previous  experiences  we  have  found  that  there  is  no  clear  cut 
* 

way  to  evaluate  forecast  algorithms  in  an  inventory  management  system.  With 
this  in  mind,  an  ad  hoc  sequential  step  wise  experiment  was  designed  where 
both  forecast  methods  and  evaluation  procedures  were  eliminated  and/or  refined 
after  each  step.  As  a  result  both  a  best  forecast  algorithm  and  a  best 
evaluation  method  may  be  determined.  The  details  of  each  step  are  as  follows: 

6.2  Step  1:  (13  forecast  algorithms,  9  statistical  error  measures) 

For  the  first  step,  the  four  major  data  sets  described  on  page 
were  used.  After  appropriate  initialization,  each  forecast  algorithm  was  used 
to  make  a  one  quarter  and  a  four  quarter  forecast  for  every  quarter  of  each 
series.  These  forecasts  were  compared  to  the  actual  demand  of  the  series  and 
the  errors  were  rolled  up  within  classes  of  series.  Various  error  measures 
were  computed  for  13  forecast  algorithms,  6  series  classes,  and  2  forecast 
horizons.  The  experimental  layout  is  as  follows: 

Data  Sets: 

Low  Dollar  Value  Non-Dynamic  (LDVNON)  consisting  of  54  items 
Low  Dollar  Value  Dynamic  (LDVDYN)  consisting  of  224  items 
High  Dollar  Value  Dynamic  (HDVDYN)  consisting  of  1199  items 
High  Dollar  Value  Non-Dynamic  (HDVNON)  consisting  of  19384  items 

Forecast  Horizons 

1  quarter 
4  quarters 

ERROR  Measures : 

Let  x^  be  the  demand  (for  the  ith  item)  in  the  jth  quarter 

F^  =  index  set  of  forecasts  for  ith  item 

7^ ^  =  cardinal  size  of  F^  (number  of  times  a  forecast  was  made) 

*=  error  (x^  -/xij),  j  t  Ft 

AVG^  “  average  demand  for  item  i 

(double  12  month  moving  average  starting  after  the 
first  non-^ero  demand) 


The  first  error  measures  considered  were  simple  averages  of  tra¬ 
ditional  measures. 

,  N  . 

MAD  -  i  Z  —  E  |e,,|  ’ 

8  i-i  ni  jfFl  13 

'  . '  (Mean  Absolute  Error)  ^ 

1^1  2 
MSE  -  —  E  ■=—  E  (E  ) 

N  i-lni  j.F4  lj 


(Mean  Square  Error) 


1  1(1  1 

BIAS  -  ±  E  —  E 

N  i-1  ni  j«F. 


Percent  Error  Measures 

The  simple  averages  give  more  weight  to  items  with  high  demand 
frequency.  Since  the  items  were  stratified  into  homogeneous  classes  it  was 
desirable  to  give  equal  weights  to  each  item  in  the  class,  hence  the  following 
percent  error  measures  were  considered. 


1  N 

AVG  %  of  Forecast  -  rr  E 

N  i-1 

jl  ; 

n  i-i 


AVG  7  of  Actual 


—  E 

ni  j-F. 


—  E 
ni  JeF. 


IfilL 

% 


(fill 
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AVG  7.  of  Both 


i  ;  i_  T 
N  i-l  ni  JcF. 


["ill 

1/2  (x^  +  "x^) 


Relative  Error  Measures 

Now  since  many  of  the  series  were  quite  variable,  the  denominator 
of  the  percent  error  measures  did  not  reflect  the  steady  state  demand  of  the 
item  hence  the  following  relative  measures  were  considered. 


RELATIVE  BIAS 


RELATIVE  MAD 


RELATIVE  MSE 


1 
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Forecast  Algorithms: 

KALMAN 

KALNEW1 

KALMANS 

KALNEWTS 

KALNEW1SMAZ 

KALNEW1 SMA3 

KALMAN SMA2 


(refer  to  Chapters  III 

IROTRIGG15-75 

IROTRIGG25-25 

REGKBNEWSMA2 

CURRENT  1794 

KALREL 

KAL2SPK 


IV,  V  for  definition) 


Series  Classifications: 

Class  1  Average  Annual  Dollar  demand  between  $0  and  $5000 
Class  2  Average  Annual  Dollar  demand  between  $5000  and  $50000 
Class  3  Average  Annual  Dollar  demand  greater  than  $50000 

Class  4  Average  Annual  Number  of  Requisitions  between  0-3 

Class  5  Average  Annual  Number  of  Requisitions  between  3-12 

Class  6  Average  Annual  Number  of  .Requisitions  greater  than  12 

6.3  Step  2:  (5  forecast  algorithms,  6  error  measures) 


This  step  consisted  of  evaluating  five  forecast  algorithms  over  a 
four  quarter  forecast  horizon.  The  data  sets  and  series  classifications  were 
the  same  as  Step  1.  Four  of  the  five  algorithms  were  the  best  ranked  ones 
from  the  previous  step  and  the  error  measures  were  those  which  appeared  most 
consistent  and/or  easiest  to  understand.  The  fifth  algorithm,  MED4 ,  was 
suggested  after  Step  1  was  completed.  The  algorithm  and  error  measures  are 
as  follows: 
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ERROR  MEASURES:  (refer  to  pages  27,  28,  29  for  definition) 

REL  MAD 
REL  MSE 
%  of  Actual 
X  of  Forecast 
X  of  Both 


Algorithms: 


KALNEW1 


KALNEW1 3MA3 
1R0TR1GG 


6.4  Step  3:  (Statistical  vs  Simulation  Evaluation) 

This  step  is  the  most  complex  of  the  three.  Samples  from  five  of 
nine  stratification  classes  were  taken  and  both  statistical  and  simulation 
analysis  were  used  to  evaluate  the  remaining  four  algorithms.  The  following 
table  gives  a  count  of  items  sampleo  in  the  3x3  dollar  demand  versus 
requisition  stratification. 


0  -  $5000 


Strat  Class 


N  -  335 


Yearly  Dollar  Demand 
i  $5000  -  $50000 


•h  ™  Analyzed 


N  -  124 


Analyzed 


N  -  98 


N  -  100 


Analyzed 


N  -  230 


Analyzed 


N  ■  64 


>  $50000 


Analyzed 
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Forecast  Algorithms: 


The  five  remaining  forecast  algorithms  considered  for  the 

analysis  were: 

KALMAN 

1794 

IROTRIGG 

MED4 

MOVD 

Moving  D  was  added  to  the  list  of  those  previous  tested  to 
determine  if  the  results  from  Cohen's  report  [3]  still  appear  valid.  That  is, 
do  forecast  algorithms  utilizing  program  data  perform  better  than  those 
forecasting  on  demand  only. 

Statistics: 

In  an  effort  to  better  relate  error  measures  to  inventory  per¬ 
formance,  the  following  procedural  changes  were  made  to  the  way  the  statistics 
were  collected. 

(1)  Forecast  only  after  a  demand;  this  is  the  only  time  a 
reorder  point  may  be  triggered  and  where  the  forecast  is  actually  used 
(alternative  would  be  periodic  review  which  wasn’t  considered). 

(2)  Use  only  the  item's  PLT  as  a  forecast  horizon;  again  this 
is  what  would  be  used  in  an  inventory  system. 

(3)  Use  a  simple  eight  quarter  average  for  the  forecast  if  the 
item  had  been  inactive  for  a  year  prior  to  the  demand  triggering  the  forecast; 
this  would  handle  the  migration  of  an  item  from  an  active  strat  class  to  an 
inactive  one  without  unduly  penalizing  the  algorithm  which  would  normally  work 
well  in  an  active  class  and  does  poorly  in  the  less  active  class. 

Along  with  these  procedural  changes,  additional  inventory  measures  were 
considered.  An  overforecast  error  in  predicting  demand  impacts  the  inventory 
control  system  differently  then  does  an  underforecast .  Overforecasts  result  in 
carrying  too  much  stock  and  increase  the  possibility  of  being  stuck  with 
obsolete  items,  whereas  underforecasts  increase  the  possibility  of  not 
satisfying  a  customer's  orders  and  in  the  case  of  the  Army  may  reduce  the 
readiness  of  a  weapon  system.  Since  there  is  not  a  natural  tradeoff  between 
these  two  types  of  errors,  the  following  separate  measures  were  developed. 
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Notation 


Underforecast  Measure 


UF(i) 


EL  (PLT) 


DiJ(PLT) 


0  R^PLT) 


is  an  estimate  of  the  number  of  requisitions  not  satisfied  for  the  i  item,  i.e. 
if  demand  is  under forecasted,  say  by  20%,  then  it  is  implied  that  20%  of  the 
requisitions  will  not  be  satisfied. 


N 

Z  UF (i) 

i=l 

N 

Z  Z 
i-1  JtF, 


(^ij (PLT) ) 


is  an  estimate  of  the  percent  of  the  total  requisition  not  satisfied  over  all  the 
items . 

The  base  period  is  the  procurement  lead  time  of  the  item  which  is  the 
quickest  time  stock  could  be  replenished  after  a  new  order  is  placed.  In  an 
underforecast  situation  the  reorder  point  will  probably  be  crossed  within  a 


Simulator 

The  final  analysis  was  done  with  the  IRO  Simulator  of  the  Army 
wholesale  supply  system,  a  description  of  which  is  found  in  Cohen  [2], 

Algorithms  are  compared  in  the  form  of  cost-performance  curves;  the  curves  are 
traced  thru  several  "X"  points  for  each  forecast  procedure,  the  lambda  (X) 
values  reflecting  an  operating  policy  which  relates  to  the  cost  of  a  backorder. 
Actual  demand  and  flying  hour  time  series  for  items  in  any  of  the  various  data 
groupings  are  used  in  particular  simulation  runs.  All  algorithms  have  the 
same  starting  conditions  prior  to  accumulating  performance  statistics.  To  do 
this  during  the  warmup  period  (2  year)  all  algorithms  utilize  an  8  quarter  moving 
average  on  demand.  Of  course,  also  during  warmup,  the  algorithms  obtain  their 
various  starting  values  where  needed. 
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To  determine  if  the  curves  are  statistically  different,  a  fixed  cost 
analysis  is  done  on  the  difference  between  the  performance  of  the  current 
policy  and  the  test  policies  for  each  item.  Histograms  of  these  differences 
are  displayed  and  statistical  tests  are  used  for  comparison.  Details  of  the 
test  procedure  are  found  in  Section  7.3. 

There  have  been  many  changes  made  to  the  simulator  since  used  in  reference 
[8].  The  most  noteworthy  are: 

(1)  Excess  costs  (projected  from  assets  above  RO  at  end  of  simulator 
run)  are  accumulated  and  averaged  into  operating  costs  only  on  items  coded 
terminal  or  obsolete  or  on  items  with  trivial  (nearly  zero  AYD)  demand.  However, 
end  of  simulation  stock  is  also  stratified  into  1  to  15  years  of  supply  over 

all  items  and  presented  as  a  simulator  output  for  each  forecast  policy. 

(2)  Previously,  forecasts  could  be  updated  in  between  quarters 
(at  time  of  buys)  using  moving,  interpolated,  quarters  formed  from  actual 
quarterly  data.  Now  forecasts  are  updated  only  on  the  actual  quarters. 

(3)  The  current  PCER  tables  utilized  in  the  VSL  module  were  previously 
adjusted  for  lead  time  by  a  theoretical  factor  of  1/SL  from  base  values.  Now, 
the  PCER  base  values  do  not  change,  in  order  to  reflect  conservatively  the 
empirical  observation  that  percentage  error  increased  with  lead  time  L. 

(4)  The  constraint  that  safety  level  be  no  more  than  the  expected 
lead  time  demand  quantity  is  lifted. 

(5)  The  simulator  now  incorporates  the  effects  of  phased  deliveries. 

(6)  The  point  estimate  of  an  item's  order  size  in  a  current  inter¬ 
polated  time  interval  (obtained  previously  from  the  interpolated  requisition 

and  demand  history)  is  now  smoothed  (averaged)  with  previous  order  size  computations. 
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CHAPTER  VII 


ANALYSIS  AND  RESULTS 

The  results  from  the  data  experiments  described  in  the  previous  chapters 
are  analyzed  in  the  next  few  sections. 

In  each  step  algorithms  were  eliminated  based  on  their  performance  when 
compared  to  the  alternative  methods  while  using  several  error  measures.  The 
final  analysis  was  performed  via  the  IRQ  simulator  on  the  resulting  five 
techniques . 


7.1  Step  1  (Rankings  based  on  9  statistical  error  measures) 

The  tables  in  this  section  summarize  the  comparative  performance 
(rankings)  of  the  algorithms  from  which  initial  screening  decisions  were  made. 
For  each  of  the  four  data  base  groups,  and  within  stratification  classes 
(dollar  demand  and  average  yearly  requisitions)  for  each  group,  the  algorithms 
are  ranked  ("1"  being  best)  based  on  values  across  the  many  error  measures 
described  in  Chapter  VI.  Rankings  are  done  for  both  1  quarter  and  4  quarter 
forecast  error  measures. 

After  a  study  of  the  ranking  patterns  in  these  tables,  KALNEW1 , 
CURRENT,  KALNEW1SMA3,  IROTRIGG  25-25*  were  chosen  for  further  statistical 
and  simulation  investigation.  KALNEV1SMA3  was  chosen  over 
KALNEW1SMA2  because  of  our  conservative  tendency  to  use  the  last  3  quarters 
(MA3)  of  data  rather  than  only  the  last  2  quarters.  Current  1794,  of  course, 
is  chosen  as  the  base  for  improvement.  MED4  was  developed  later  in  our  study. 


The  "25-25"  refers  to  the  a^,  values  in  hendredths. 
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7*2  Step  2  (Ranks  from  four  quarter  error  measures) 

Four  tables  are  presented  in  this  section,  one  for  each  of  the 
four  item  groupings  -  LDV  dynamic  and  non-dynamic  items,  HDV  dynamic  and 
non-dynamic  items.  Each  table  presents  rankings  of  the  five  candidate  items 
for  two  stratifications,  by  annual  dollar  demand  and  by  annual  demand  fre¬ 
quency  (requisitions) .  The  relative  performance  rankings  in  terms  of  6 
error  measures  are  tabulated;  it  should  be  noted  that  all  but  the  last  are 
relative  or  "percent"  error  measures. 

The  algorithm  MED4  was  statistically  evaluated  and  ranked  only  for 
the  measures  REL  MAD  and  REL  MSE.  These  two  measures,  incidently,  are 
the  most  consistent,  in  the  sense  that  their  rank  orderings  most  frequently 
agree  with  a  consensus  rank  ordering  across  all  the  measures  in  a  stratifica¬ 
tion  class. 

A  pattern  of  some  note  in  the  tables:  1794  and  KALNF.W1  are  often 
ranked  closely  ("paired”)  compared  to  the  KALNEW1SMA3  -  IROTRIGG  pair.  The 
latter  pair  tend  to  perform  well  in  less  active  classes  (1,  II,  V,  VI), 
their  tracking  signals  reacting  to  fluctuations ,  while  the  1794-KALNEW1 
algorithms  weight  more  past  history  and  hence  perform  well  on  items  with  more 
stable  D/H  values,  i.e.  the  active  classes  (II,  III,  V,  VI). 

There  is  no  dominant  algorithm  across  all  strat  classes  and  tables. 
In  Step  3  we  focus  upon  the  statistical  and  simulator  performances  of  the 
algorithms  in  a  three  by  three  stratification. 
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7.3  Step  3  (Simulation  and  final  statistical  results) 

In  this  section  the  performance  results  via  simulated  cost  effec¬ 
tiveness  and  statistical  error  measures  are  analyzed.  The  final  four 
algorithm  candidates,  MED4,  1794,  KALI  (formerly  coded  KALNEW1) ,  Trigg 
(IROTRIGG) ,  along  with  MOVD  (moving  average  on  demands)  were  used  to  compute 
forecasts  in  various  simulation  runs  over  five  classes  (active  items)  of  the 
3x3  data  stratification  described  on  page  29  .  The  simulated  cost  - 
performance  curves  are  captured  in  pages  44  -  50  where  performance  is  measured 
in  terms  of  the  average  of  the  time  weighted  backorders  as  opposed  to  average 
days  wait  which  was  reported  in  the  previous  studies.  (This  transformation 
of  performance  measures  has  no  impact  on  the  simulation  results  but  does 
make  the  conyarative  analysis  easier,  ref  [fc]). 

Tha  table  on  page  51  contains  the  ‘statistics  from  the  various  error  measures 
for  each  of  the  five  data  classes  and  for  both  a  one  quarter  and  PLT  forecast 
horizon.  The  second  part  of  the  table  compares  the  within  class  ranks  of 
the  statistical  measures  and  the  simulation  results  which  were  ranked  at  a 
fixed  cost  as  described  in  the  next  section  on  Final  Analysis. 
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Since  there  are  obvious  discrepancies  between  the  simulation  ranks  and 
statistical  ranks  a  decision  had  to  be  made  as  to  what  results  should  be 
used  for  the  final  analysis.  Much  experimentation  was  done  with  the  simulator 
but  inconsistencies  with  the  statistics  continued  to  plague  the  results.  A 
discussion  with  Prof  Mueller  of  the  University  of  Ghent  (Belgium)  revealed  that 
work  being  done  on  an  unpublished  PhD  thesis  using  Monte  Carlo  methods  indicate 
that  minimum  mean  squared  error  forecast  techniques  are  not  necessarily  optimal 
methods  when  applied  to  inventory  management  models.  This  fact  along  with  our 
belief  that  the  IRO  simulator  best  represents  the  Army  management  system  com¬ 
pelled  us  to  use  the  simulated  results  for  our  final  decisions. 

7.4  Final  Results  (Evaluation  of  Simulation  Results) 

In  an  effort  to  determine  if  there  is  a  statistical  difference  between 
the  cost-performance  curves  generated  by  the  simulator  the  following  fixed 
cost  analysis  was  employed,  details  of  which  are  in  [6]. 

For  each  class  of  items,  a  fixed  (current)  cost  is  computed  by  running  the 
simulator  using  the  standard  forecast  algorithm  (1794)  and  the  A  value  the 
Commodity  Command  presently  uses.  The  resulting  cost  represents  the  inventory 
cost  presently  incurred  to  manage  the  class  of  items.  Also  during  this  run, 
the  performance  of  each  item  is  arrayed  in  a  data  file  for  future  analysis. 

For  the  alternative  policies  (forecast  algorithms)  several  As  are  used 
to  generate  the  cost-performance  curve  for  each  policy.  Using  the  shape 
of  this  curve,  a  spline  technique  is  used  to  determine  the  performance  of  each 
item  using  the  alternative  policy  at  the  fixed  cost.  Distribution'- (histograms) 
of  item  performance  for  each  policy  at  the  current  cost  are  found  in 
Appendix  A. 

To  better  measure  the  difference  between  the  alternative  methods  and 
the  standard,  the  difference  between  the  individual  item  performance  for  the 
alternative  and  the  standard  were  computed.  (It  is  shown  in  the  basic 
statistical  literature  that  these  differences  will  be  less  variable  due  to 
the  elimination  of  extraneous  effects  and  will  measure  only  the  difference 
in  methods.)  The  distributions  of  these  differences  are  found  in  Appendix  B. 

Statistically  testing  for  zero  means  for  each  of  these  difference 
distributions  is  equivalent  to  testing  for  a  difference  between  the  standard 
(1794)  curve  and  the  alternatives  as  plotted  on  pages  44  to  50.  The  results  of 
these  tests  are  captured  on  the  next  page  where  95%  confidence  intervals  are 
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displayed  for  each  distribution  of  differences  for  each  class  of  items  tested. 
(Note  if  these  intervals  do  not  contain  zero,  then  the  null  hypothesis  of 
zero  mean  is  rejected  at  an  a  level  of  5%)  The  rankings  found  on  page  52 
taken  from  this  table  by  looking  at  the  mid-point  of  each  interval  and 
ranking  them  from  left  to  right. 

Findings 

(1)  Using  the  IRO  simulator  as  described  in  this  report,  there  is  no 
statistical  difference  in  the  simulated  performance  for  each  of  the  five 
stratification  classes  for  the  following  comparison. 

KALMAN  vs  1794 
IROTRIGG  vs  1794 
MED4  vs  1794 

(2)  MOVD  performed  worse  than  1794  and  the  other  alternative  algorithms 
for  the  more  active  items  -  Class  8  and  9 . 

Note:  MOVD  was  the  only  algorithm  not  using  program  data. 

(3)  There  was  no  difference  between  MOVD  and  1794  for  less  active  classes 
7,  6,  5,  4. 
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CHAPTER  VIII 


CONCLUSION 

8 . 1  Findings 

In  an  earlier  report  by  Orr  [8]»  the  candidate  of  final  choice  for 
forecasting  was  a  Kalman  filter  algorithm.  This  choice  was  based  upon  its 
dominance  in  statistical  forecast  accuracy  and  its  savings  over  the  current 
method  as  projected  from  cost-performance  curves  produced  from  simulator  runs. 
There  was  little  difference  amongst  the  algorithms'  curves  for  LDV  items, 
and  so  most  of  the  savings  was  driven  by  HDV  items. 

From  the  current  viewpoint  it  appears  that  the  savings  might  not  have 
been  statistically  significant.  At  the  least,  the  algorithm's  simulated  per¬ 
formance  has  not  been  robust  against  the  changes  that  have  occurred  (see  below) 
since  that  report  (although  its  relative  forecast  accuracy  has  held  up) . 
Presently  we  find  "Kalman"  working  well  on  some  of  our  groupings  of  items 
and  the  current  (1794)  method  (among  other  algorithms)  working  well  on  other 
items.  The  improved  statistical  tools  [6]  we  use  now  show,  in  any  case,  that 
no  algorithm's  cost-performance  curve  is  significantly  better  than  another's  - 
for  active  items.  In  several  cases  a  few  items  can  influence  the  performance 
rankings . 

It  is  not  possible  to  isolate  the  impact  of  individual  changes  made  since 
the  earlier  report  when  comparing  the  differing  results.  However,  such  changes 
were : 

a.  The  data  base  was  extended  from  7  to  11  years  and  more  items  were 
captured.  The  last  7  years  of  the  11  were  used  in  the  current  analysis,  so 
much  of  the  Vietnam  era  from  the  earlier  time  series  was  not  influential. 

b.  It  is  possible  that  a  different  small  group  of  items  might  now  be 
driving  the  HDV  savings. 

c.  The  statistical  analysis  program  was  overhauled  to  be  more  flexible 
and  capture  various  error  measures. 

d.  Changes  in  the  simulator  were  made.  Several  of  these  could  narrow 
the  potential  difference  in  performance  amongst  the  algorithms,  e.g.,  excess 
cost  savings  were  accumulated  only  for  some  items,  a  standard  moving  average 
forecast  superseded  all  algorithms  in  periods  of  very  little  activity,  fore¬ 
cast  updates  were  made  only  after  quarters  with  demand. 

e.  Some  changes  to  the  algorithms  themselves  were  made.  Theoretical 

adjustments  were  made  to  the  "Kalman"  to  produce  several  versions;  also,  the 
"k"  parameters  were  updated  to  reflect  the  later  7  years  of  history  that  we  re¬ 
used.  56 


8.2  Postscript 


This  table  consolidates  statistical  results  of  one  of  the  more  meaningful 
error  measures,  MAD/AYD,  for  the  four  algorithms  -  by  data  group  and  by 
requisition  class.  This  relative  error  when  multiplied  by  an  algorithm's 
current  forecast  of  average  yearly  demand,  A  YD,  yields  an  estimate  of  the  mean 
absolute  deviation  (error)  in  a  year's  demand.  The  theory  and  formulas  in 
[9]  can  be  used  to  convert  this  estimate  to  a  variance  of  lead  time  demands, 
the  latter  a  necessary  variable  in  computing  safety  levels  in  the  VSL  EOQ 
module  of  CCSS . 

It  is  apparent  that  the  choice  of  a  final  algorithm  dictates  the  percent 
errors  to  be  used  in  VSL  EOQ.  The  present  PCER  tables  in  that  module  should 
be  superseded  by  an  expanded,  refined  version  of  these  tables  by  requisition 
class  (and  perhaps  by  dollar  demand).  The  MAD/AYD  for  the  inactive  class 
(0-3  requisitions)  would  be  based  upon  current  work  on  algorithms  for  inactive 
items,  and  not  upon  those  values  in  this  current  table,  which  for  some  entries 
are  suspect. 

8.3  Recommendations 

For  active  items  (greater  than  3  requisitions  a  year)  the  current  (1794) 
forecast  procedure  should  not  be  replaced. 

The  percent  error  (PCER)  tables  in  the  VSL  EOQ  module  should  be  over¬ 
hauled  using  the  statistical  byproducts  of  this  current  research  and  of  the 
inactive  item  research. 

A  plan  for  consolidating  the  current  algorithm  (for  active  items),  the 
pending  algorithm  (for  inactive  items,  including  NSO  6  insurance)  and  the 
consequent  lead  time  demand  variance  procedure  should  be  instituted. 
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HISTOGRAMS  OF  INDIVIDUAL  ITEM  PERFORMANCE  AS  DETERMINED 

BY  SIMULATOR 
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APPENDIX  B 


HISTOGRAM  OF  DIFFERENCES  BETWEEN  INDIVIDUAL  PERFORMANCE 
OF  ALTERNATIVE  ALGORITHMS  AND  STANDARD  (1794) 
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1 _  HQ,  Dept  of  the  Army,  (DASG-HCL-P) ,  Wash.,  DC  20314 
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of  California,  Berkeley,  CA  94720 

1 _  Dr.  Jack  Muckstadt,  Dept  of  Industrial  Engineering  &  Operations 

Research,  Upson  Hall,  Cornell  University,  Ithaca,  NY  14890 
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